The hierarchical prediction network: towards a neural theory of grammar acquisition

We present a biologically inspired computational framework for language processing and grammar acquisition, called the hierarchical prediction network (HPN). HPN fits in the tradition of connectionist models, but it extends their power by allowing for a substitution operation between the nodes of the network. This, and its hierarchical architecture, enable HPN to function as a full syntactic parser, able to emulate context free grammars without necessarily employing a discrete notion of categories. Rather, HPN maintains a graded and topological representation of categories, which can be incrementally learned in an unsupervised manner. We argue that the formation of topologies, that occurs in the learning process of HPN, offers a neurally plausible explanation for the categorization and abstraction process in general. We apply HPN to the task of semi-supervised ‘grammar induction’ from bracketed sentences, and demonstrate how a topological arrangement of lexical and phrasal category representations successfully emerges.